IVS-605 Run ClamAV and file magic detection to erase potentially harmful content #248
Conversation
aothms
changed the title
|
@aothms added missing pip and some extra output for DJ admin console |
| <Link href={`${FETCH_PATH}/api/download/${row.id}`} underline="hover" onClick={evt => evt.stopPropagation()}> | ||
| {'Download file'} | ||
| </Link> | ||
| { |
There was a problem hiding this comment.
My idea was already to strip the file contents so that downloading is harmless https://github.com/buildingSMART/validate/pull/248/files#diff-5db999c346dafe19127402f8a9220bff8cafead114de27b3e971a88d55e4b0dcR162
But thanks for all the additions to this 👍
Yes so far I did not because of the continuous memory consumption for having the signatures (I assume) always in memory. By loading them as part of the task we kind of naturally follow the memory consumption pattern of the other tasks. But I agree, the slowdown is hard to justify |
Really basic.
This depends on clamav for detecting large zips (by setting limits on max extracted contents and by making encountering such limits a failure).
Also depends on
filetypeto extract content type from file contents.Current check is not really interesting (maybe a specific purpose detection for zip bombs make more sense) and loading signatures in clamav takes some time (chose not to run the daemon because of mem usage). But usage of clamav could also be extended in the future.
The file contents are erased when mimetype is detected (such as PNG) from the file contents bytes. The file is not deleted because that would cause integrity errors when a file with the same name is uploaded again (apparently the file postfixes for uniqueness are based on filesystem not db)